System and method for constraining air traffic communication (atc) transcription in real-time

ABSTRACT

Systems and methods are provided for the selection of a speech model for automatic speech recognition during the runtime of a transcription system, the system includes an event detector to determine one of a number of flight events that include flight plan changes and phase transitions based on data received from a set of inputs; an intelligent keyword generator to collate a set of keywords associated with the flight plan information and to generate a wordlist in response to a determination by the event detector of flight plan changes or flight phase transitions; and a processor to determine whether the wordlist is covered by a current speech model implemented in the automatic speech recognition wherein if the wordlist is not covered by the current speech model, then the processor to select a pre-built speech model that covers the wordlist for use as the current speech model in the automatic speech recognition.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to India Provisional Patent ApplicationNo. 202211021488, filed Apr. 11, 2022, the entire content of which isincorporated by reference herein.

TECHNICAL FIELD

The subject matter described herein relates generally to vehiclesystems, and more particularly, embodiments of the subject matter relateto avionics systems and methods to constrain a model for AutomaticSpeech Recognition (ASR) used in transcribing cockpit communications inreal-time.

BACKGROUND

Even though datalink technologies are employed in in-flightcommunications, a majority of communications between an Air TrafficController (ATC) and a pilot is through speech using cockpit radiochannels. Speech recognition applications are used in aircraft systemsfor transcription of ATC-pilot communications as well as for providingother commands, control, and automatic terminal information service(ATIS) information.

The ATC transcription to transcribe cockpit communications for therequired high accuracy is data-intensive requiring a lot of keywordsstored to perform accurate transcription in the aviation domain. This isbecause, in the aviation domain, there are a host of unique keywords andacronyms that require storage for purposes of recognition whileexecuting an Automatic Speech Recognition (ASR) model. Such acronyms andkeywords may include airport names, Navigation Aid Systems (NAVAIDs),procedure names, Standard Instrument Departure Routes, and StandardArrival Routes (SID/STARs), or route-specific keywords such as airways,fixes, airspace, waypoints, Non-Directional Beacon (NDB)/VeryHigh-Frequency Omni-Directional Range (VOR)/Terminal Radar ApproachControl Facilities (TRACON)/Very High-Frequency Omni-Directional RadioRange Tactical Air Navigation Aid (VORTAC), Visual Flight Rule (VFR)point, USER waypoints, call signs, and taxiways that are unique to theaviation domain. The requirement of having amounts of unique keywordterminology stored for accuracy in transcription causes increases inmemory usage and latency when executing an ASR model by a transcriptionsystem.

It is desirable to overcome the drawbacks posed by the requirements ofhaving larger memories and latencies caused by voluminous amounts ofunique keywords in the ASR model and to enable an ATC transcriptionapplication that is configured to be executed locally in a cockpitenvironment with virtually no or limited external cloud support.

It is desirable for the ASR model to be executable on an edge device byadding a set of constraints on the compute platform when implementingthe ASR model as with an edge-based solution there are little or limitedcomputation capabilities and limited memory available.

It is desirable to enable an ASR model and language processing system inan aviation domain that at least overcomes the drawbacks caused byrequirements of terminologies of voluminous non-standard Englishkeywords like waypoints, fixes, navaids, custom fixes, andcryptic/unusual pronunciation with variations that may impact theaccuracy of the ASR model in usage.

It is desirable to provide methods and systems that realize improvementsin ASR model performance of accuracy and latency by constraints in theASR model size that include constraints to speech models which areoptimized to perform better for recognition accuracy and latency.

It is desirable to limit the usage of storing large vocabularies thatresult in increases in the search paths causing inferencing as well asadding memory requirements for loading and causing increasing time fortextual inference.

It is desirable to balance the requirements of storing unique words foraccuracy with the detrimental effects of storing and parsing too manyunique words that can be detrimental to speech recognition accuracy.

It is desirable to provide methods and systems to dynamically detect anevent that aids in switching or generation of constrained modelsincluding all required keywords in each flight stage to improve speechrecognition performance.

Other desirable features and characteristics of the methods and systemswill become apparent from the subsequent detailed description and theappended claims, taken in conjunction with the accompanying drawings andthe preceding background.

BRIEF SUMMARY

Aircraft systems and related operating methods are provided. In oneembodiment, a transcription system with a selectable speech model usedin automatic speech recognition is provided. The system includes anevent detector configured to determine one of a number of flight eventsthat comprise flight plan changes and a flight phase transition based onevent data received from a set of inputs; an intelligent keywordgenerator in operable communication with the event detector andconfigured to collate a set of keywords associated with at least flightplan information to generate a wordlist in response to a determinationby the event detector of the flight plan changes or the flight phasetransition wherein the wordlist contains keywords associated with theflight plan changes and the flight phase transition; and a processor inoperable communication with the intelligent keyword generator andconfigured to determine, based on the wordlist from intelligent keywordgenerator, whether the wordlist is covered by a current speech modelimplemented in the automatic speech recognition of the transcriptionsystem, wherein if the wordlist is not covered by the current speechmodel, then the processor is further configured to communicate with adatabase storing one or more pre-built speech models to select apre-built speech model that covers the wordlist for use as the currentspeech model in the automatic speech recognition of the transcriptionsystem.

In at least one exemplary embodiment, the transcription system furtherincludes if the pre-built speech model is not available that covers thewordlist from the intelligent keyword generator, then the processor isfurther configured to generate a new speech model for use as the currentspeech model with the automatic speech recognition of the transcriptionsystem to enable coverage of the wordlist by the current speech modeland constraint of the current speech model to at least the flight planchanges or the flight phase transition.

In at least one exemplary embodiment, the processor is furtherconfigured to: determine the coverage of the wordlist from theintelligent keyword generator by comparison of the keywords in thewordlist contained in each of the one or more pre-built speech modelsstored in the database.

In at least one exemplary embodiment, the intelligent keyword generatoris further configured to: collate the keywords used in communication inthe flight plan changes or the flight phase transition for comparison ofkeyword coverage in each of the one or more pre-built speech modelsstored in the database.

In at least one exemplary embodiment, the event detector is furtherconfigured to receive the event data from the set of inputs, wherein theset of inputs comprises a first input of pilot input, the second inputof data input from one or more aircraft systems that include FlightManagement System (FMS) data, the third input of Air Traffic Control(ATC) clearance data, and the fourth input of log data generated by oneor more checklists or pilot logs.

In at least one exemplary embodiment, the set of inputs is configured ina hierarchy by the processor to determine a flight phase change orflight phase transition with the pilot input given the highest value.

In at least one exemplary embodiment, the processor is furtherconfigured to: implement a plurality of checks to determine whether datafrom the first, second, third, or fourth input triggers thedetermination by the event detector of the flight plan change or theflight transition for re-selecting of the current speech model in use bythe transcription system.

In another exemplary embodiment, a method of implementing automaticspeech recognition during the runtime of a transcription system isprovided. The method includes determining, by an event detector, one ofa number of flight events that comprise flight plan changes and a flightphase transition based on event data received from a set of inputs;collating, by an intelligent keyword generator, a set of keywordsassociated with at least flight plan information for generating awordlist in accordance with the flight plan changes or the flight phasetransition determined by the event detector, wherein the wordlistcontains keywords associated the flight plan changes and the flightphase transition; determining, by a processor based on the wordlist fromthe intelligent keyword generator, whether the wordlist is covered by acurrent speech model implemented in automatic speech recognition of thetranscription system; and in response to a determination that thewordlist is not covered by the current speech model, selecting, by theprocessor by communicating with a database storing one or more pre-builtspeech models, the pre-built speech model covering the wordlist.

In at least one exemplary embodiment, the method includes in response tothe determination that the pre-built speech model covering the wordlistfrom the intelligent keyword generator is not available, generating bythe processor, a new speech model for use as the current speech modelfor the automatic speech recognition of the transcription system forenabling coverage of the wordlist by the current speech model and forconstraining the current speech model to at least the flight planchanges or the flight phase transition.

In at least one exemplary embodiment, the method includes determiningcoverage, by the processor, of the wordlist from the intelligent keywordgenerator by comparison of the wordlist to the keywords contained ineach of the one or more pre-built speech models stored in the database.

In at least one exemplary embodiment, the method includes collating, bythe processor, keywords of the wordlist used in communication in theflight plan changes or the flight phase transition for comparison ofkeyword coverage in each of the one or more pre-built speech modelsstored in the database.

In at least one exemplary embodiment, the method includes receiving, bythe event detector, the event data from the set of inputs comprising afirst input of pilot input, the second input of data input from one ormore aircraft systems that include Flight Management System (FMS) data,the third input of Air Traffic Control (ATC) clearance data, and thefourth input of log data generated by one or more checklists or pilotlogs.

In at least one exemplary embodiment, the method includes configuring,by the processor, a hierarchy for determining a flight phase change orflight phase transition with the pilot input given the highest value.

In at least one exemplary embodiment, the method includes implementing,by the processor, a plurality of checks for determining whether datafrom the first, second, third, or fourth input triggers thedetermination by the event detector of the flight plan change or theflight transition for re-selecting of the current speech model in use bythe transcription system.

In yet another exemplary embodiment, at least one non-transientcomputer-readable medium having instructions stored thereon that areconfigurable to cause at least one processor to perform a method forselection of a speech model in automatic speech recognition duringruntime of a transcription system is provided. The method includesdetermining, by the at least one processor, one of a number of flightevents comprising flight plan changes and a flight phase transitionbased on event data received from a set of inputs; collating, by the atleast one processor, a set of keywords associated with at least flightplan information for generating a wordlist in accordance with flightplan changes or the flight phase transition determined by the eventdetector, wherein the wordlist contains keywords associated the flightplan changes and the flight phase transition; determining, by the atleast one processor based on the wordlist from the intelligent keywordgenerator, whether the wordlist is covered by a current speech modelimplemented in the automatic speech recognition of the transcriptionsystem; and in response to a determination that the wordlist is notcovered by the current speech model, selecting, by the at least oneprocessor by communication with a database storing one or more pre-builtspeech models, a pre-built speech model covering the wordlist.

In at least one exemplary embodiment, the method includes in response tothe determination that the pre-built speech model covering the wordlistis not available, generating by at least one processor, a new speechmodel for use as the current speech model for the automatic speechrecognition of the transcription system for enabling coverage of thewordlist by the current speech model and for constraining the currentspeech model to at least the flight plan changes or the flight phasetransition.

In at least one exemplary embodiment, the method includes determiningcoverage, by at least one processor, of the wordlist by comparison ofthe wordlist to the keywords contained in each of the one or morepre-built speech models stored in the database.

In at least one exemplary embodiment, the method includes collating, byat least one processor, keywords of the wordlist used in communicationin the flight plan changes or the flight phase transition for comparisonof keyword coverage in each of the one or more pre-built speech modelsstored in the database.

In at least one exemplary embodiment, the method includes receiving, byat least one processor, the event data from the set of inputs comprisinga first input of pilot input, the second input of data input from one ormore aircraft systems that include Flight Management System (FMS) data,the third input of Air Traffic Control (ATC) clearance data, and thefourth input of log data generated by one or more checklists or pilotlogs.

In at least one exemplary embodiment, the method includes implementing,by at least one processor, a plurality of checks for determining whetherdata from the first, second, third, or fourth input triggers thedetermination by the event detector of the flight plan change or theflight transition for re-selecting of the current speech model in use bythe transcription system.

Furthermore, other desirable features and characteristics of the subjectmatter described herein will become apparent from the subsequentdetailed description and the appended claims, taken in conjunction withthe accompanying drawings and the preceding background.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will hereinafter be described in conjunction withthe following figures, wherein like numerals denote like elements, andwherein:

FIG. 1 depicts an exemplary embodiment of a processing system to selecta speech model based on event detection of a transcription system whichmay be utilized with a vehicle, such as an aircraft in accordance withan embodiment;

FIG. 2 depicts an exemplary diagram of multiple inputs used in eventdetection for the selection of a speech model of the transcriptionsystem in accordance with an embodiment;

FIGS. 3A and 3B depict exemplary diagrams of the flow process of thespeech model selection system based on event detection changes enablingthe use of a current speech model with constraints by the transcriptionsystem in accordance with an embodiment; and

FIG. 4 depicts an exemplary diagram of a flow process of determiningcoverage of keywords by the ASR model of speech model selection systemin accordance with an embodiment.

DETAILED DESCRIPTION

The following detailed description is merely exemplary in nature and isnot intended to limit the subject matter of the application and usesthereof. Furthermore, there is no intention to be bound by any theorypresented in the preceding background, brief summary, or the followingdetailed description.

The Air Traffic Controller (ATC) is typically involved with voicecommunications between a pilot or crewmember onboard the variousaircraft within controlled airspace. The ATC and the pilot are often inconstant communication over a Voice Channel or the CPDLC throughout theflight. The ground-to-air communications include clearance, information,or requests for message elements. The ATC to pilot communications haveseveral limitations including but not restricted to miscommunication,transmission errors, misinterpreted voice instructions, ambiguouscommunications, non-standard phraseology that can cause an increasedpilot overload of operational tasks when listening to ATC amidst flyingthe aircraft.

In in-flight operations, there exists a large number of terminologyassociated with different arrival procedures, approaches, and departureprocedures. The ATC-pilot communication in a customary dialog willcontain a general class of keywords that includes call sign specifickeywords that allow both parties in the communication dialog to know oridentify the flight to prevent or at least limit confusion with respectto flight specific instructions. In embodiments, the ATC-pilotcommunication may include a realm of different types of messages thatinclude a clearance instruction, an advisory message, or query(question) issued by the controller to the pilot as well as readbackrequests, reports, or queries issued by the pilot to the controller. Ineither case, there may be specific keywords used in the respectiveflight phase transition and flight phase that increase (or contribute)to the vocabulary keyword list (or keyword corpus) of a speech modelthat is used for transcription of a dialog exchange.

In embodiments, to enhance the speech model, it is desirable toimplement the usage of a keyword list with unique keywords that aredependent on the flight phase or transition in the flight phase whichcan be used also as a basis to necessitate changes in a vocabulary setor keyword list of unique keywords contained in a current ASR model inusage. In this case, the current ASR model with a configured keywordlist enables increases in accuracy of transcription of the voicedcommunications in the respective flight phase transition or flightphase.

In various exemplary embodiments, the present disclosure describesmethods and systems that automatically provide a display of theextracted text of clearance or communication of the flight phase ortransition to the pilot or other flight personnel for review and toprevent miscommunications in ATC instructions and other flightoperations.

In various exemplary embodiments, methods and systems described areassociated with a voice-activated flight deck that enables speechrecognition or brings a speech recognition system into the cockpit.

In various exemplary embodiments, the methods and systems providecapabilities associated with command and control and transcribing of ATCconversations. In various exemplary embodiments, the present disclosuredescribes methods and systems that reduce the pilot's workload byimproving a current ASR model by the selection of the ASR model in usebased on runtime inputs and at least changes in the flight plan. Inembodiments, the ASR models are configured (or constrained) with uniquekeywords used in the speech models for transcriptions based on inputsfrom users, ATC communications, and changes in the flight phase.

In various exemplary embodiments, the present disclosure describesmethods and systems that reduce frequency congestion experienced byreducing the need between the pilot and ATC to make repeated requestsfor clearance communications based on improvements of ASR models bykeyword data associated with a flight phase. Other residual benefits toexisting and planned cockpit functionality may include enhancements toplayback of radio communications and real-time transcription of radiocommunications which are more contextual and intelligent.

For purposes of explanation, the subject matter is primarily describedherein in the context of aircraft operating in controlled airspace;however, the subject matter described herein is not necessarily limitedto aircraft or avionic environments, and in alternative embodiments, maybe implemented equivalently for ground operations, marine operations, orotherwise in the context of other types of vehicles and travel spaces.

FIG. 1 depicts an exemplary embodiment of a transcription processingsystem 100 operating during the runtime of a transcription system whichmay be utilized with a vehicle, such as an aircraft in accordance withan embodiment. FIG. 1 in an exemplary embodiment, shows a transcriptionprocessing system 100 includes, without limitation, a display system 40,one or more user input devices (user input 46), communication systems 5,a clearance assistance system 66, a flight management system (FMS) 70,one or more avionic systems 60, and a data storage 50 suitablyconfigured to support the operation of multiple sets of speech models(speech model 62), as described in greater detail below.

In an embodiment, as shown in FIG. 1 , a clearance instruction orclearance command may be received from the Air Traffic Control (ATC) viathe communication system 5 of an aircraft that may be configured toinclude several communication channels or interfaces of a data link 7, aCOM radio 10, and pilot audio 20 for sending and receiving commands andinstructions in different flight phases or flight transitions.

In an embodiment, in the case of an ATC audio message (e.g., ATCclearance audio message), an aeronautical operational control (AOC)message, and/or a pilot voice audio, the various audio type messages areprocessed to a transcription system (transcription module 30) forspeech-to-text conversion and for displaying the transcribed text on thedisplay device 40 for visual notification to the pilot. Thetranscription module 30 can be implemented with one or more differentASR models (i.e., speech model 29) for enhanced speech transcription ofthe performance of the functions associated with speech-to-textconversion. In an implementation, a speech to text converter (speechengine 23) may be used with the transcription module 30 to receive thevoice and audio input of the radio communications (from communicationsystems 5) and may store the transcribed text for display during theruntime of the speech-to-text conversion process.

In exemplary embodiments, with further reference to FIG. 1 , theaircraft communication system 5 may receive and send voicecommunications between the pilot and the controller, other recordedcockpit audio, and data via the data link 7. The output from thecommunication system 5 is received by the transcription module 30 thatmay be configured with a speech engine 23 that includes various softwareapplications 31 to enhance the accuracy of speech-to-text extraction andconversion by a natural language processing (NLP) unit 27 of voiceconversations that occur between the pilot and the controller during theflight. The NLP unit 27 as described is configured to implement a speechengine 23 that uses a speech model (ASR model) 41 to perform functionssuch as morphological segmentation, entity recognition, conversion ofchunks of text into formal representations, tagging parts of speech,parsing, relationship extraction, and sentiment analysis of audiosegment parts of the flight conversations between the pilot andcontrollers. The flight conversations as an example may includeclearance messages that are voiced between the pilot and the controller.

In an exemplary embodiment, the speech engine 23 of the NLP unit 27 isconfigured with a set of speech analysis applications (various softwareapplications 31) that include an application for spotting keywords andsentence segments in voice conversations; an application thatcategorizes sentences based on priority; an application to categorizesentence segments from the application, and an application to determinea category of the message captured in the voice conversations and anapplication to determine the flight context on the captured message.

In exemplary embodiments, an output from the transcription module 30 isreceived by a clearance assistance system 66, various avionic systems60, and the flight management system (FMS) 70 of the aircraft. Forexample, the output from the NLP unit 27, after the speech analysis, issent to the clearance assistance system 66 so that certain context-basedpriority tagging operations can be performed based, for example, on thepriority of the words, and filtering or segmenting of the words andphrases. Once completed, the output is sent to various cockpit displaysystems 40 and/or configured as an audio output for communication on theaircraft audio systems to the pilot.

In exemplary embodiments, the speech engine 23 may be configured toinclude an ASR model 41 that is switched between multiple ASR pre-builtmodels 62 (at data storage 55) based on coverage of unique keywordsidentified in a keyword list by an intelligent Keyword Generator (iKG)54. The iKG 54 with a smart module 42 monitors in real-time or nearreal-time coverage of the unique keywords in-flight phases andidentifies all required specific keywords at given times. In animplementation, the smart module 42 can trigger events to generateconstrained models dynamically if the prebuilt models that cover thekeywords are not found. The constrained model (i.e., the ASR model 41)which is the current ASR model in use will cover all the keywords andunique keywords generated by the iKG 54 in each flight phase andtranscribe incoming messages in the particular flight phase or flighttransition for better speech recognition performance.

In another embodiment, this feature of selection of a constrained modelcan be configured in an (ATC transcription) app 85 on a mobile device 80connected wirelessly with the avionic systems 60 with user input enabledin a settings page or tab of the app 85. The user can view thetranscription quality on the display system 40 and if not satisfied cantrigger the reconfiguration process of the ASR model 41 (via the app 85)to generate or replace the current speech model using current flightparameters or other user input. Even the event trigger can be enabled ordisabled using the configuration page of the app 85. This will enablethe user to reconfigure, re-generate or constrain an existing ASR model41 even when there is no change in a flight phase or transition orevent-triggered or detected by the event detector 50.

In embodiments, the parameters configured in the constraint models canbe selected in alternative ways such as based on the geographic region.In this case, the geographic region may be separated or divided intomultiple zones or segmented by states or cities, other geographicalboundaries, accent-based boundaries, and sectors or Air Traffic ControlCenters (ARTCC) centers on which a constraint model may be selected. Inanother embodiment, constraint ASR models (selected from data storage55) can be constrained or selected based on an inputted flight plan (viainput devices 46) or even on a more granular level to a flight phase ofthe flight plan.

In embodiments, the constraint ASR model can be configured in accordancewith various prebuilt models (from data storage 55) that are of adefinitive size and are stored with a wordlist including all thekeywords that are required to generate the appropriate constraint modelfor use. In implementations, as an example, the wordlist can be splitinto two parts of a first list containing generic English keywords usedin communication/dialogue, and a second list containing aviationdomain-specific keywords.

In embodiments, an event detector 50 is configured to process event dataand to trigger the iKG 54 to generate new wordlists based on the eventdata that is used as a basis to determine changes in flight routes,flight phases, and flight transitions. The event data, as an example,can enable the event detector 50 and/or the smart module 41 to cause thetriggering of a wordlist generation by determining deviations of anactive primary flight plan, a switch to a secondary, an alternate flightplan, or a new route assignment.

In embodiments, the phase transition may also be provided as input tothe ATC transcription app (app 85) by the user. In the case of a manualuser entry, the input would be given the highest priority as a finalflight phase is determined based on manual user entry.

FIG. 2 depicts an exemplary diagram 200 with the event detector 50 andthe components of the Natural language processing (NLP) unit 27 of FIG.1 in accordance with exemplary embodiments. The event detector 50 iscombined with the intelligent keyword generator (iKG) 54 where the iKG54 provides the wordlist for selecting the speech model by the smartmodel 42 via the model selector or generator 43. The iKG 54 can generatewordlists that include unique words, acronyms, messages, and terminologyassociated with the detected transition or event change in flightoperations. The event detector 50 is configured to detect a multitude ofdifferent events that can include complex phase transitions or lesscomplex changes that are used in the basis of selection of the speechmodel with constraints. The event detector 52 determines the change ortransition in the flight plan or other flight event changes that affectthe speech model in use by inputs received from the ATC clearancemessages, pilot inputs, and inputs from flight management systems andother avionic systems in communication with the event detector 50. In animplementation, the transition or a flight event change may be detectedfrom data from the avionic systems, pilot input, or transcribed andanalyzed clearance messages.

In an implementation, a hierarchy can be configured to prioritizeapplied events. For example, a higher priority may be ascribed to inputreceived by the pilot or other user's manual/voice input to thetranscription system.

In an embodiment, the manual user input may be designated with thehighest priority as the final flight phase is customarily determinedbased on the manual (pilot) input of a flight plan received. Forexample, the event detector 50 may determine the phase of the flight anddisplay the current active phase in the display (i.e., navigationdisplay in the cockpit or display via a user interface of a mobiledevice) to the user for verification and/or notification. Otherimplementations include the determination of the phase transition orevent based on the data from the FMS, or the FMS (i.e., FMS 70 of FIG. 1) itself determining the phase transition or upcoming event changes andnotifying the event detector 50 directly.

This active phase event or phase transition event will be provided tothe ATC transcription app (app 85 of FIG. 1 ) and the iKG 54 if it isconnected to a flight deck display or FMS or connected to other avionicssubsystems.

In embodiments, the phase transition may be determined throughRadiofrequency handover clearances or the type of clearances received bythe ATC transcription app. The template or type of clearance for ownshipor traffic may also be indicative of the current phase of flight.

In embodiments, the phase transition may be determined by scanning thepilot-filled checklist. As per standard operating procedure, the pilotneeds to fill the checklist based on the current phase before thetransition to the new phase. This may indicate the current phase if anATC transcription app is connected to a checklist application or otheravionics subsystem that enables the sharing of this event information.

In embodiments, the speech engine may be supported or embedded in aclient on a mobile platform such as found in a mobile device like aniPAD® or other tablet or smartphone. In this case, to enable the speechengine 23 (in FIG. 1 ), the speech model 41 must be configured toexecute on a thin or limited local memory and must include all thekeywords in the particular flight phase or another flight-related eventto enable a sufficient accuracy of transcription in the flightoperation. In implementations, the speech model 41 is enabled based on aconstraint-sized model in accordance with the flight plan or based onthe phase of the flight in the flight plan and can be selectable from aspeech model database 47 connected to the smart module 42.

In embodiments, the speech model can be configured with acousticcharacteristics which are speech specific including related to thelanguage and/or to the verbiage of the user and speaker; for example,including characteristics related to the pronunciation of the commonwords and the unique words. The language corpus or vocabulary set canalso be trimmed or constrained to particular flight events, phases ortransitions so that the acoustic model used in the NPL processing wouldinclude all possible or most of the keywords required for the ATCmessage transcription.

In embodiments, the active phase event or phase transition event isprovided to the ATC transcription app (ex. app 85 of FIG. 1 ) if it isconnected to the flight deck display or FMS, or other avionicssubsystems. The phase transition can be determined throughRadiofrequency handover clearances or the type of clearances received bythe ATC transcription app. The template or type of clearance for ownshipor traffic is indicative of the current phase of flight. The phasetransition can be also determined by scanning a pilot-filled checklistthat is received as input to the transcription system. For example, perstandard operating procedure, since a pilot is customarily required tofill the checklist based on a respective current phase before thetransition to the new phase, the phase transition can be determinedbased on the checklist status and information included in the currentflight phase. The checklist information also may be used as an indicatorabout the current phase and transition to the next phase when the ATCtranscription app is connected to input data from the checklist or otherlinked avionics subsystems that are configured to share phase and eventinformation.

FIGS. 3A and 3B depict exemplary flow diagrams of the transcriptionprocess with the event detection and iKG generation of FIGS. 1-2 inaccordance with exemplary embodiments. In FIG. 3A, at step 303, input isreceived of pilot updates and flight plans or changes in flight routes.At step 305, the input is processed for a current active flight plan onthe ATC transcription app or by flight deck avionics. In step 310, adetermination is made by the event detector of the transcriptionprocessing system if the current flight plan has changed. If the flightplan has been changed, then at step 315 the iKG is initiated and isconfigured to fetch new keywords related to the updated flight plan andnew routes. Also, various keywords are collated by the iKG, and thekeywords which are collated may be related to the airport, NAVAIDs,procedure names, route-specific keywords such as airways. Alternately,if the current flight plan is determined to have not changed at step310, then the process flow proceeds to institute a four-step checkingprocess to check data from a set of inputs of the transcription systemfor flight phase, transition, and event changes. The set of inputs withdata checked includes data from a pilot input and clearance messages,event changes, handover commands, and checklists.

In an embodiment, at the first step of the four-step process at step320, a determination is made of the phase of flight by pilot input databy the ATC transcription system. That is based on input 323 from thepilot, and apps that are configured with enabled checkbox radio buttonsto generate input data. In an implementation, at step 325, once adetermination is made if a pilot input has triggered the ATCtranscription app then at step 320 a determination is made of the flightphase. At step 330 another check is performed of the four-step processof detecting whether a flight transition is occurring. If the flighttransition is not detected, then at step 335, data of events and changesin FMS cockpit avionics or display are analyzed to determine the flightphase. The output of this analysis is checked at step 340 of anotherstep of the four-step process, to determine if a transition in theflight phase can be detected. If a flight phase transition is notdetected, then at step 345, input from ATC transcribedmessages/clearances via input 347 is analyzed and processed for afrequency change, radio handover commands, or determinations of categorytemplates for other ATC clearances for flight phase determinations. Atstep 350, another check is performed on whether there is a transition inthe phase of the flight that can be detected. If there is no detectionof the flight phase transition, then the input 357 from the pilotchecklist is processed at step 355 which is again checked for atransition at step 360 of a phase change in the flight. If there is notransition detected at this final step, then at step 365, the currentspeech model is used by the transcription system for the ATCtranscription.

If at step 340, after processing the event data and analysis of flightdata from aircraft avionic systems, it is determined that there isdetected a transition in the flight phase; then the process proceeds tostep 385 in FIG. 3B to retrieve new keywords related to the new flightphase. For example, this step may include collating all keywordsavailable for a particular flight transition or flight phase includingwaypoints and call signs, and other specific keywords at step 385. As anexample, the new keywords may be obtained from NAV databases 375,generic English keywords, callsign databases 380, and airport chartdatabase 370 in communication with the transcription system. At step395, after the retrieval of the new keywords, a determination is made asto whether the keywords are covered in a pre-build model? Thisdetermination is made by a comparison of the new keyword list (listcomposed of keywords retrieved), and the pre-built models of word listsincluded in a database of prebuilt speech models 390. If the new keywordlist is covered then at step 400 the pre-built model is loaded orswitched to the ASR in use. If the new keywords are not covered in anyof the pre-built models in the database of prebuilt speed models 390,then a new speech model at step 405 is generated and the new speechmodel is used at step 410 in transcribing the ATC transcription. Also,the new speech model is added at step 415 to the database of pre-builtmodels 390 for later use, and for updating other speech models ifrequired.

FIG. 4 is an exemplary flowchart 400 for improving real-timetranscription using a selective Automatic Speech Recognition (ASR) modelwhen transcribing pilot-Air Traffic Control (ATC) communications by thecockpit transcription system in accordance with various embodiments. InFIG. 4 , at step 425, a method of implementing automatic speechrecognition during the runtime of a transcription system is initiated.At step 430, input is received from a set of inputs that include a firstinput of pilot input, the second input of data input from one or moreaircraft systems that include Flight Management System (FMS) data, thethird input of Air Traffic Control (ATC) clearance data, and the fourthinput of log data generated by one or more checklists or pilot. In animplementation, the set of inputs is configured in a hierarchy with themanual or pilot input given the most value when determining the nextstep of a change by a flight event. At step 435, the inputs areprocessed by an event detector to determine one of a number of changesbased on a set of flight events that include flight plan changes and aflight phase transition based on event data received from the set ofinputs. At step 440, if there is a determined flight event change, thenan intelligent keyword generator determines a set of keywords associatedwith the flight plan information and then generates a wordlist inaccordance with the flight plan changes or the flight phase transitiondetermined by the event detector. The keywords in the wordlist arecollated by the intelligent keyword generator and then at step 445, aprocessor is implemented to determine whether the wordlist is covered bya current speech model that is implemented in the automatic speechrecognition of the transcription system in use. At step 450, in responseto a determination that the wordlist is not covered by the currentspeech model, the processor communicates with a database storing one ormore pre-built speech models and selects a pre-built speech model thatcovers the wordlist.

At step 455, if there is no pre-built speech model found that covers thewordlist, then the processor generates a new speech model for use as thecurrent speech model for the automatic speech recognition of thetranscription system. This new speech model will therefore enablecoverage of the wordlist and replace the current speech model in use.The new speech model is constrained to at least the flight plan changesor the flight phase transition.

For the sake of brevity, conventional techniques related to air trafficcontrol, aviation communications, aviation terminology, flightmanagement, route planning and/or navigation, aircraft procedures,aircraft controls, and other functional aspects of the systems (and theindividual operating components of the systems) may not be described indetail herein. Furthermore, the connecting lines shown in the variousfigures contained herein are intended to represent exemplary functionalrelationships and/or physical couplings between the various elements. Itshould be noted that many alternative or additional functionalrelationships or physical connections may be present in an embodiment ofthe subject matter.

The subject matter may be described herein in terms of functional and/orlogical block components and with reference to symbolic representationsof operations, processing tasks, and functions that may be performed byvarious computing components or devices. It should be appreciated thatthe various block components shown in the figures may be realized by anynumber of hardware components configured to perform the specifiedfunctions. For example, an embodiment of a system or a component mayemploy various integrated circuit components, e.g., memory elements,digital signal processing elements, logic elements, look-up tables, orthe like, which may carry out a variety of functions under the controlof one or more microprocessors or other control devices. Furthermore,embodiments of the subject matter described herein can be stored on,encoded on, or otherwise embodied by any suitable non-transitorycomputer-readable medium as computer-executable instructions or datastored thereon that, when executed (e.g., by a processing system),facilitate the processes described above.

The foregoing description refers to elements or nodes or features being“coupled” together. As used herein, unless expressly stated otherwise,“coupled” means that one element/node/feature is directly or indirectlyjoined to (or directly or indirectly communicates with) anotherelement/node/feature, and not necessarily mechanically. Thus, althoughthe drawings may depict one exemplary arrangement of elements,additional intervening elements, devices, features, or components may bepresent in an embodiment of the depicted subject matter. Also, certainterminology may be used in the following description for reference only,and thus are not intended to be limiting. For example, terms such as“first,” “second,” and other such numerical terms may be utilized torefer to or distinguish between different elements or structures withoutimplying a sequence or order unless indicated by the context.

While at least one exemplary embodiment has been presented in theforegoing detailed description, it should be appreciated that a vastnumber of variations exist. It should also be appreciated that theexemplary embodiment or exemplary embodiments are only examples, and arenot intended to limit the scope, applicability, or configuration of thesubject matter in any way. Rather, the foregoing detailed descriptionwill provide those skilled in the art with a convenient road map forimplementing an exemplary embodiment of the subject matter. It should beunderstood that various changes may be made in the function andarrangement of elements described in an exemplary embodiment withoutdeparting from the scope of the subject matter as set forth in theappended claims. Accordingly, details of the exemplary embodiments orother limitations described above should not be read into the claimsabsent a clear intention to the contrary.

What is claimed is:
 1. A transcription system with a selectable speechmodel used in automatic speech recognition comprising: an event detectorconfigured to determine one of a number of flight events that compriseflight plan changes and a flight phase transition based on event datareceived from a set of inputs; an intelligent keyword generator inoperable communication with the event detector and configured to collatea set of keywords associated with at least flight plan information togenerate a wordlist in response to a determination by the event detectorof the flight plan changes or the flight phase transition wherein thewordlist contains keywords associated with the flight plan changes andthe flight phase transition; and a processor in operable communicationwith the intelligent keyword generator and configured to determine,based on the wordlist from the intelligent keyword generator, whetherthe wordlist is covered by a current speech model implemented in theautomatic speech recognition of the transcription system, wherein if thewordlist is not covered by the current speech model, then the processoris further configured to communicate with a database storing one or morepre-built speech models to select a pre-built speech model that coversthe wordlist for use as the current speech model in the automatic speechrecognition of the transcription system.
 2. The transcription system ofclaim 1, further comprising: if the pre-built speech model is notavailable that covers the wordlist from the intelligent keywordgenerator, then the processor is further configured to generate a newspeech model for use as the current speech model with the automaticspeech recognition of the transcription system to enable coverage of thewordlist by the current speech model and constraint of the currentspeech model to at least the flight plan changes or the flight phasetransition.
 3. The transcription system of claim 2, wherein theprocessor is further configured to: determine the coverage of thewordlist from the intelligent keyword generator by comparison of thekeywords in the wordlist contained in each of the one or more pre-builtspeech models stored in the database.
 4. The transcription system ofclaim 3, wherein the intelligent keyword generator is further configuredto: collate the keywords used in communication in the flight planchanges or the flight phase transition for comparison of keywordcoverage in each of the one or more pre-built speech models stored inthe database.
 5. The transcription system of claim 4, wherein the eventdetector is further configured to: receive the event data from the setof inputs, wherein the set of inputs comprises a first input of pilotinput, a second input of data input from one or more aircraft systemsthat include Flight Management System (FMS) data, a third input of AirTraffic Control (ATC) clearance data, and a fourth input of log datagenerated by one or more checklists or pilot logs.
 6. The system ofclaim 5, wherein the set of inputs are configured in a hierarchy by theprocessor to determine a flight phase change or flight phase transitionwith the pilot input given a highest value.
 7. The transcription systemof claim 6, wherein the processor is further configured to: implement aplurality of checks to determine whether data from the first, second,third, or fourth input triggers the determination by the event detectorof the flight plan change or the flight transition for re-selecting ofthe current speech model in use by the transcription system.
 8. A methodof implementing automatic speech recognition during runtime of atranscription system, the method comprising: determining, by an eventdetector, one of a number of flight events that comprise flight planchanges and a flight phase transition based on event data received froma set of inputs; collating, by an intelligent keyword generator, a setof keywords associated with at least flight plan information forgenerating a wordlist in accordance with the flight plan changes or theflight phase transition determined by the event detector, wherein thewordlist contains keywords associated with the flight plan changes andthe flight phase transition; determining, by a processor based on thewordlist from the intelligent keyword generator, whether the wordlist iscovered by a current speech model implemented in automatic speechrecognition of the transcription system; and in response to adetermination that the wordlist is not covered by the current speechmodel, selecting, by the processor by communicating with a databasestoring one or more pre-built speech models, the pre-built speech modelthat covers the wordlist.
 9. The method of claim 8, further comprising:in response to the determination that the pre-built speech modelcovering the wordlist from the intelligent keyword generator is notavailable, generating by the processor, a new speech model for use asthe current speech model for the automatic speech recognition of thetranscription system for enabling coverage of the wordlist by thecurrent speech model and for constraining the current speech model to atleast the flight plan changes or the flight phase transition.
 10. Themethod of claim 9, further comprising: determining coverage, by theprocessor, of the wordlist from the intelligent keyword generator bycomparison of the wordlist to the keywords contained in each of the oneor more pre-built speech models stored in the database.
 11. The methodof claim 10, further comprising: collating, by the processor, keywordsof the wordlist used in communication in the flight plan changes or theflight phase transition for comparison of keyword coverage in each ofthe one or more pre-built speech models stored in the database.
 12. Themethod of claim 11, further comprising: receiving, by the eventdetector, the event data from the set of inputs comprising a first inputof pilot input, a second input of data input from one or more aircraftsystems that include Flight Management System (FMS) data, a third inputof Air Traffic Control (ATC) clearance data, and a fourth input of logdata generated by one or more checklists or pilot logs.
 13. The methodof claim 12, further comprising: configuring, by the processor, ahierarchy for determining a flight phase change or flight phasetransition with the pilot input given a highest value.
 14. The method ofclaim 13, further comprising: implementing, by the processor, aplurality of checks for determining whether data from the first, second,third, or fourth input triggers the determination by the event detectorof the flight plan change or the flight transition for re-selecting ofthe current speech model in use by the transcription system.
 15. Atleast one non-transient computer-readable medium having instructionsstored thereon that are configurable to cause at least one processor toperform a method for selection of a speech model in automatic speechrecognition during runtime of a transcription system, the methodcomprising: determining, by the at least one processor, one of a numberof flight events comprising flight plan changes and a flight phasetransition based on event data received from a set of inputs; collating,by the at least one processor, a set of keywords associated with atleast flight plan information for generating a wordlist in accordancewith flight plan changes or the flight phase transition determined bythe event detector, wherein the wordlist contains keywords associatedthe flight plan changes and the flight phase transition; determining, bythe at least one processor based on the wordlist from the intelligentkeyword generator, whether the wordlist is covered by a current speechmodel implemented in the automatic speech recognition of thetranscription system; and in response to a determination that thewordlist is not covered by the current speech model, selecting, by theat least one processor by communication with a database storing one ormore pre-built speech models, a pre-built speech model covering thewordlist.
 16. The method of claim 15, further comprising: in response tothe determination that the pre-built speech model covering the wordlistis not available, generating by the at least one processor, a new speechmodel for use as the current speech model for the automatic speechrecognition of the transcription system for enabling coverage of thewordlist by the current speech model and for constraining the currentspeech model to at least the flight plan changes or the flight phasetransition.
 17. The method of claim 16, further comprising: determiningcoverage, by the at least one processor, of the wordlist by comparisonof the wordlist to the keywords contained in each of the one or morepre-built speech models stored in the database.
 18. The method of claim17, further comprising: collating, by the at least one processor,keywords of the wordlist used in communication in the flight planchanges or the flight phase transition for comparison of keywordcoverage in each of the one or more pre-built speech models stored inthe database.
 19. The method of claim 18, further comprising: receiving,by the at least one processor, the event data from the set of inputscomprising a first input of pilot input, a second input of data inputfrom one or more aircraft systems that include Flight Management System(FMS) data, a third input of Air Traffic Control (ATC) clearance data,and a fourth input of log data generated by one or more checklists orpilot logs.
 20. The method of claim 19, further comprising:implementing, by the at least one processor, a plurality of checks fordetermining whether data from the first, second, third, or fourth inputtriggers the determination by the event detector of the flight planchange or the flight transition for re-selecting of the current speechmodel in use by the transcription system.